In order to solve the problem that Scale Invariant Feature Transform (SIFT) algorithm has low matching accuracy and long time consuming in image matching, an improved image matching algorithm based on grid motion statistical feature, namely RANSAC-GMS, was proposed. Firstly, the image was pre-matched by Oriented FAST and Rotated BRIEF (ORB) algorithm and Grid-based Motion Statistics (GMS) was used to support the estimator to distinguish the correct matching points from the wrong matching points. Then, an improved RANdom SAmple Consensus (RANSAC) algorithm was used to filter the feature points according to the distance similarity between the matching points, and an evaluation function was used to reorganize the filtered new datasets to eliminate the mismatching points. The experiments were carried out on Oxford standard image library and images taken in reality. Experimental results show that the average matching accuracy of the proposed algorithm in image matching is over 91%. Compared with algorithms such as GMS, SIFT and ORB, the near-scene matching accuracy and the far-scene matching accuracy of the proposed algorithm are improved by 16.15 percentage points and 3.56 percentage points respectively. The proposed algorithm can effectively eliminate mismatching points and achieve further improvement of image matching accuracy.